Aspects Of Natural Language Processing Research Articles

Cross-Domain Aspect Based Sentiment Analysis

Abstract: Sentiment analysis, a crucial aspect of natural language processing, faces significant challenges in diverse domains. This research focuses on cross-domain aspect-based sentiment analysis, aiming to enhance accuracy across various fields. We propose a novel framework tailored for this purpose, addressing existing limitations. Through comprehensive evaluations on diverse datasets, our approach demonstrates improved efficacy compared to established models. The discussion highlights the strengths and limitations of our framework, contributing valuable insights to the field of sentiment analysis. Our research emphasizes the need for domain-specific considerations in sentiment analysis, offering practical implications for real-world applications. In conclusion, this study advances our understanding of cross-domain sentiment analysis challenges and provides a practical solution for improved accuracy in diverse domains

Research on Term Recognition in the Field of English Linguistics Based on Deep Learning Algorithms

Linguistic recognition, a fundamental aspect of natural language processing, has seen significant advancements with the integration of deep learning algorithms. his paper presents a novel approach leveraging deep learning algorithms, specifically an optimized Long Short-Term Memory (LSTM) model, for term recognition tasks in English linguistics. This paper presents a novel approach leveraging deep learning algorithms, particularly Weighted Genetic Optimization Deep Learning (WGODL), for term recognition tasks in English linguistics. his hybrid approach not only enhances model accuracy but also improves robustness and generalization, enabling accurate term recognition even in noisy or ambiguous linguistic contexts. The weighted genetic optimization mechanism strategically prioritizes relevant features during the learning process, further enhancing the model's ability to discern domain-specific terms. Additionally, qualitative analysis highlights the interpretability of learned representations, shedding light on the underlying linguistic structures captured by the model. By harnessing the sequential nature of language data, the proposed model integrates LSTM units with optimization techniques to effectively capture contextual dependencies and semantic relationships within text corpora. Through extensive experimentation on diverse linguistic datasets, our optimized LSTM model achieves superior term recognition accuracy, surpassing traditional methods by 8.3%.

Keystroke Logging for Activity-Monitoring using Python

Cyberwarfare is observed very frequently as always some or the other country is targeting to ruin its enemy country by hacking confidential data from vital computer systems. This has led to dangerous international conflicts. Hence, to avoid illicit entry of other than military person or a government official several tools are being used today as spyware. Keyloggers are one of the prominent tools which are used in today’s world to obtain secret or confidential data of a legitimate and contradictory a malicious user too. These keyloggers are advantageous and taken up positively for monitoring employee productivity, for law enforcement and the search for evidence of the crime. While it’s negative illegitimate use includes data theft and passwords. The keylogger is today witnessed as a malicious attack and is looked upon as a security threat. But every coin has two sides. Keylogger actually helps in avoiding several security breaches and also aids in detecting several crimes across the net world followed by other fellow countries. This fact has motivated to write this paper and as a consequence, an experimental analysis too was carried out in order to conclude that keyloggers’ log file helps identify the person by analyzing proper pattern of the words entered in the file. This paper focuses majorly on the aspect of natural language processing, where a log file obtained thru keylogger software is thoroughly processed via the algorithm as described in the paper. The results yielded a fair understanding of the results obtained as one can easily identify the words used and on the basis of that can also know the type of person on the other end with his ideas, malicious one or of a legal kind

Semantic Similarity Caculating based on BERT

The exploration of semantic similarity is a fundamental aspect of natural language processing, as it aids in comprehending the significance and usage of vocabulary present in a language. The advent of pre-training language models has significantly simplified the process of research in this field. This article delves into the methodology of utilizing the pre-trained language model, BERT, to calculate the semantic similarity among Chinese words. In order to conduct this study, we first trained our own model using the bert-base-chinese pre-trained model. This allowed us to acquire the word embeddings for every single word, which served as the basis for calculating semantic similarity. Essentially, word embeddings are vector-based depictions of words that encapsulate word’s significance and surroundings, allowing for the measurement of the semantic similarity between words. Next, we executed a sequence of experiments to assess the efficiency of the BERT model in managing semantic similarity tasks within the Chinese language. The results were encouraging, as the BERT model demonstrated remarkable performance in these tasks. Furthermore, it was observed that the BERT model outperformed traditional methods in terms of performance and generalization capabilities. This study, therefore, underscores the potential of the BERT model in natural language processing, particularly in the Chinese language. This emphasizes the model’s capacity to accurately calculate semantic similarity, paving the way for its widespread adoption in related fields.

Enhanced negation handling for sentiment analysis on Twitter using deep neural networks

<span>Sentiment analysis is a tool to identify and measure the emotion in a piece of text. Negation handling is an important aspect of natural language processing (NLP) for Twitter data. It is a process of correctly interpreting the sentences containing negation words, such as, "never", "no", "neither" and so on. Negation words are used in machine learning to express negative sentiment or indicate the absence of something. In this article, a negation handling technique using deep learning models. Artificial neural networks (ANNs) and convolutional neural networks (CNNs) for classification is proposed. The system is evaluated on SemEval-2017 dataset. The classification performance is improved by using ANN and CNN on the negative tweets. The study aims to improve the classification accuracy by considering negation words in the text. The paper compares the performance of ANNs and CNNs in handling negation words and evaluates them on the tweets data. This study provides insights into the effectiveness of using deep learning techniques for negation handling in sentiment analysis and highlights the importance of considering negation words in the text for improved sentiment analysis performance. The proposed negation strategy attains a superior performance accuracy over machine learning models by preventing misclassified tweets.</span>

News Classification and Categorization with Smart Function Sentiment Analysis

Search engines are tools used to find information on the Internet. Since the web has a plethora of websites, the engine queries the majority of active sites and builds a database organized according to keywords utilized in the search. Because of this, when a user types a few descriptive words on the home page of the search engine, the search function lists websites corresponding to these keywords. However, there are some problems with this search approach. For instance, if a user wants information about the word Jaguar, most search results are animals and cars. This is a polysemic problem that forces search engines to always provide the most popular but not the most relevant results. This article presents a study of using sentiment technology to help news classification and categorization and improve the classification accuracy. We have introduced a smart search function embedded into a search engine to tackle polysemic issues and record relevant results to determine their sentimentality. Therefore, this study presents a topic that involves several aspects of natural language processing (NLP) and sentiment analysis for news categorization and classification. A web crawler was used to collect British Broadcasting Corporation (BBC) news across the Internet, carried out preprocessing of text by using NLP, and applied sentiment analysis methods to determine the polarity of the processed text data. The sentimentality represents negative, positive, or neutral polarities assigned by the sentiment analysis algorithms. The research utilized the BBC news site to collect different information using a web crawler and a database to explore the sentimentality of BBC news. The natural language toolkit (NLTK) and BM25 indexed and preprocessed patterns in the database. The experimental results depict the proposed search function surpassing normal search with an accuracy rate of 85%. Moreover, the results show a negative polarity of BBC news using the Sentistrength algorithm. Furthermore, the Valence Aware Dictionary and sEntiment Reasoner (VADER) was the best-performing sentiment analysis model for news classification. This model obtained an accuracy of 85% using data collected with the proposed smart function.

A hybrid part-of-speech tagger with annotated Kurdish corpus: advancements in POS tagging

Abstract With the rapid growth of online content written in the Kurdish language, there is an increasing need to make it machine-readable and processable. Part of speech (POS) tagging is a critical aspect of natural language processing (NLP), playing a significant role in applications such as speech recognition, natural language parsing, information retrieval, and multiword term extraction. This study details the creation of the DASTAN corpus, the first POS-annotated corpus for the Sorani Kurdish dialect. The corpus, containing 74,258 words and thirty-eight tags, employs a hybrid approach utilizing the bigram hidden Markov model in combination with the Kurdish rule-based approach to POS tagging. This approach addresses two key problems that arise with rule-based approaches, namely misclassified words and ambiguity-related unanalyzed words. The proposed approach’s accuracy was assessed by training and testing it on the DASTAN corpus, yielding a 96% accuracy rate. Overall, this study’s findings demonstrate the effectiveness of the proposed hybrid approach and its potential to enhance NLP applications for Sorani Kurdish.

Enhancing Deep Learning-Based Sentiment Analysis Using Static and Contextual Language Models

Sentiment Analysis (SA) is an essential task of Natural Language Processing and is used in various fields such as marketing, brand reputation control, and social media monitoring. The various scores generated by users in product reviews are essential feedback sources for businesses to discover their products' positive or negative aspects. However, it takes work for businesses facing a large user population to accurately assess the consistency of the scores. Recently, automated methodologies based on Deep Learning (DL), which utilize static and especially pre-trained contextual language models, have shown successful performances in SA tasks. To address the issues mentioned above, this paper proposes Multi-layer Convolutional Neural Network-based SA approaches using Static Language Models (SLMs) such as Word2Vec and GloVe and Contextual Language Models (CLMs) such as ELMo and BERT that can evaluate product reviews with ratings. Focusing on improving model inputs by using sentence representations that can store richer features, this study applied SLMs and CLMs to the inputs of DL models and evaluated their impact on SA performance. To test the performance of the proposed approaches, experimental studies were conducted on the Amazon dataset, which is publicly available and considered a benchmark dataset by most researchers. According to the results of the experimental studies, the highest classification performance was obtained by applying the BERT CLM with 82% test and 84% training accuracy scores. The proposed approaches can be applied to various domains' SA tasks and provide insightful decision-making information.

Instructor-assisted question classification system using machine learning algorithms with N-gram and weighting schemes

One aspect of natural language processing, text classification, has become necessary in the educational domain due to the increasing number of students and the COVID-19 outbreak. The advent of the devastating pandemic and the need to remain safe have surged the discussions around online learning and integrated modules in teaching and learning. In this study, we employed machine learning to develop an automatic instructor-assisted question classification module for learning management systems. In selecting the best classifier, the conventional and the ensemble machine learning algorithms were compared using the tenfold and the fivefold cross-validation techniques. In addition, the N-gram feature selection mechanism and three weighting schemes were evaluated for performance enhancement. The detailed analysis indicates that the ensemble algorithms outperform the conventional ones with decreasing accuracy as the N-gram size increases. For all compared algorithms, the AdaBoost (SVM) ensemble algorithm has the highest accuracy of 78.55% for Unigram (TP, TF, TF-IDF). In addition, the AdaBoost (SVM) emerged with the highest F1-score of 0.782, whiles the ensemble Bagging (RF) algorithm had the highest ROC value of 0.955 for Unigram (TP).

Pre-Training MLM Using Bert for the Albanian Language

Abstract Knowing that language is often used as a classifier of human intelligence and the development of systems that understand human language remains a challenge all the time (Kryeziu & Shehu, 2022). Natural Language Processing is a very active field of study, where transformers have a key role. Transformers function based on neural networks and they are increasingly showing promising results. One of the first major contributions to transfer learning in Natural Language Processing was the use of pre-trained word embeddings in 2010 (Joseph, Lev, & Yoshua, 2010). Pre-trained models like ELMo (Matthew, et al., 2018) and BERT (Delvin, et al., 2019) are trained on large corpora of unlabeled text and as a result learning from text representations has achieved good performance on many of the underlying tasks on datasets from different domains. Pre-training in the language model has proven that there has been an improvement in some aspects of natural language processing, based on the paper (Dai & Le, 2015). In present paper, we will pre-train BERT on the task of Masked Language Modeling (MLM) with the Albanian language dataset (alb_dataset) that we have created for this purpose (Kryeziu et al., 2022). We will compare two approaches: training of BERT using the available OSCAR dataset and using our alb_dataset that we have collected. The paper shows some discrepancies during training, especially while evaluating the performance of the model.

Evaluation of Chinese Natural Language Processing System Based on Metamorphic Testing

A natural language processing system can realize effective communication between human and computer with natural language. Because its evaluation method relies on a large amount of labeled data and human judgment, the question of how to systematically evaluate its quality is still a challenging task. In this article, we use metamorphic testing technology to evaluate natural language processing systems from the user’s perspective to help users better understand the functionalities of these systems and then select the appropriate natural language processing system according to their specific needs. We have defined three metamorphic relation patterns. These metamorphic relation patterns respectively focus on some characteristics of different aspects of natural language processing. Moreover, on this basis, we defined seven metamorphic relations and chose three tasks (text similarity, text summarization, and text classification) to evaluate the quality of the system. Chinese is used as target language. We extended the defined abstract metamorphic relations to these tasks, and seven specific metamorphic relations were generated for each task. Then, we judged whether the metamorphic relations were satisfied for each task, and used them to evaluate the quality and robustness of the natural language processing system without reference output. We further applied the metamorphic test to three mainstream natural language processing systems (including BaiduCloud API, AliCloud API, and TencentCloud API), and on the PWAS-X datasets, LCSTS datasets, and THUCNews datasets. Experiments were carried out, revealing the advantages and disadvantages of each system. These results further show that the metamorphic test can effectively test the natural language processing system without annotated data.

A Study on the journey of Natural Language Processing models: from Symbolic Natural Language Processing to Bidirectional Encoder Representations from Transformers

In this digital era, Natural language Processing is not just a computational process rather it is a way to communicate with machines as humanlike. It has been used in several fields from smart artificial assistants to health or emotion analyzers. Imagine a digital era without Natural language processing is something which we cannot even think of. In Natural language Processing, firstly it reads the information given and after that begins making sense of the information. After the data has been properly processed, the real steps are taken by the machine throwing some responses or completing the work. In this paper, I review the journey of natural language processing from the late 1940s to the present. This paper also contains several salient and most important works in this timeline which leads us to where we currently stand in this field. The review separates four eras in the history of Natural language Processing, each marked by a focus on machine translation, artificial intelligence impact, the adoption of a logico-grammatical style, and an attack on huge linguistic data. This paper helps to understand the historical aspects of Natural language processing and also inspires others to work and research in this domain.

Sentiment Analysis for Thai Language in Hotel Domain Using Machine Learning Algorithms

Sentiment analysis is one of the most frequently used aspects of Natural Language Processing (NLP), which utilizes the polarity classification of reviews expressed at the aspect, sentence or document level. Several businesses and organizations utilize this technique to improve production, as well as employee and service efficiency. However, the users’ reviews in our study were expressed in an unstructured data form, which contained spelling errors, leading to complex classifications for both the users and the machine. To solve the problem, a supervised technique of Machine Learning (ML) algorithms can be applied to the data extraction, where classification polarity can be categorized into a positive, negative or neutral class. In this research, we compared nine ML algorithms to determine the most suitable ML algorithm for creating sentiment polarity classification of customer reviews in Thai, which is a low-resource language. The dataset was collected manually from two online agencies (Agoda.com and Booking.com) utilizing a special Thai language. We employed 11 preprocessing steps to clean and handle the large amount of noise data. Next, the Delta TF-IDF, TF-IDF, N-Gram, and Word2Vec techniques were applied to convert the text reviews into vectors, processed with different ML algorithms, to determine sentiment polarity classification and to make accurate comparisons. All ML algorithms were evaluated for sentiment polarity classification with ten-fold cross-validation, with which to compare the values of recall, precision, F1-score and accuracy. The experiment results show that the Support Vector Machine (SVM) using the Delta TF-IDF technique was the best ML algorithm for polarity classification of hotel reviews in the Thai language with the highest accuracy of 89.96%. The results of this research can be applied as the tool for small and medium-sized enterprises within the field of sentiment analysis of the Thai language in the hotel domain.

LUX: Smart Mirror with Sentiment Analysis for Mental Comfort.

As COVID-19 solidifies its presence in everyday life, the interest in mental health is growing, resulting in the necessity of sentiment analysis. A smart mirror is suitable for encouraging mental comfort due to its approachability and scalability as an in-home AI device. From the aspect of natural language processing (NLP), sentiment analysis for Korean lacks an emotion dataset regarding everyday conversation. Its significant differences from English in terms of language structure make implementation challenging. The proposed smart mirror LUX provides Korean text sentiment analysis with the deep learning model, which examines GRU, LSTM, CNN, Bi-LSTM, and Bi-GRU networks. There are four emotional labels: anger, sadness, neutral, and happiness. For each emotion, there are three possible interactive responses: reciting wise sayings, playing music, and sympathizing. The implemented smart mirror also includes more-typical functions, such as a wake-up prompt, a weather reporting function, a calendar, a news reporting function, and a clock.

Decoding Strategies for Improving Low-Resource Machine Translation

Pre-processing and post-processing are significant aspects of natural language processing (NLP) application software. Pre-processing in neural machine translation (NMT) includes subword tokenization to alleviate the problem of unknown words, parallel corpus filtering that only filters data suitable for training, and data augmentation to ensure that the corpus contains sufficient content. Post-processing includes automatic post editing and the application of various strategies during decoding in the translation process. Most recent NLP researches are based on the Pretrain-Finetuning Approach (PFA). However, when small and medium-sized organizations with insufficient hardware attempt to provide NLP services, throughput and memory problems often occur. These difficulties increase when utilizing PFA to process low-resource languages, as PFA requires large amounts of data, and the data for low-resource languages are often insufficient. Utilizing the current research premise that NMT model performance can be enhanced through various pre-processing and post-processing strategies without changing the model, we applied various decoding strategies to Korean–English NMT, which relies on a low-resource language pair. Through comparative experiments, we proved that translation performance could be enhanced without changes to the model. We experimentally examined how performance changed in response to beam size changes and n-gram blocking, and whether performance was enhanced when a length penalty was applied. The results showed that various decoding strategies enhance the performance and compare well with previous Korean–English NMT approaches. Therefore, the proposed methodology can improve the performance of NMT models, without the use of PFA; this presents a new perspective for improving machine translation performance.

Keystroke Logging: Integrating Natural Language Processing Technique to Analyze Log Data

Cyberwarfare is observed very frequently as always some or the other country is targeting to ruin its enemy country by hacking confidential data from vital computer systems. This has led to dangerous international conflicts. Hence, to avoid illicit entry of other than military person or a government official several tools are being used today as spyware. Keyloggers are one of the prominent tools which are used in today’s world to obtain secret or confidential data of a legitimate and contradictory a malicious user too. These keyloggers are advantageous and taken up positively for monitoring employee productivity, for law enforcement and the search for evidence of the crime. While it’s negative illegitimate use includes data theft and passwords. The keylogger is today witnessed as a malicious attack and is looked upon as a security threat. But every coin has two sides. Keylogger actually helps in avoiding several security breaches and also aids in detecting several crimes across the net world followed by other fellow countries. This fact has motivated to write this paper and as a consequence, an experimental analysis too was carried out in order to conclude that keyloggers’ log file helps identify the person by analyzing proper pattern of the words entered in the file. This paper focuses majorly on the aspect of natural language processing, where a log file obtained thru keylogger software is thoroughly processed via the algorithm as described in the paper. The results yielded a fair understanding of the results obtained as one can easily identify the words used and on the basis of that can also know the type of person on the other end with his ideas, malicious one or of a legal kind.

Identification of Sarcasm in Textual Data: A Comparative Study

Abstract Purpose Ever increasing penetration of the Internet in our lives has led to an enormous amount of multimedia content generation on the internet. Textual data contributes a major share towards data generated on the world wide web. Understanding people’s sentiment is an important aspect of natural language processing, but this opinion can be biased and incorrect, if people use sarcasm while commenting, posting status updates or reviewing any product or a movie. Thus, it is of utmost importance to detect sarcasm correctly and make a correct prediction about the people’s intentions. Design/methodology/approach This study tries to evaluate various machine learning models along with standard and hybrid deep learning models across various standardized datasets. We have performed vectorization of text using word embedding techniques. This has been done to convert the textual data into vectors for analytical purposes. We have used three standardized datasets available in public domain and used three word embeddings i.e Word2Vec, GloVe and fastText to validate the hypothesis. Findings The results were analyzed and conclusions are drawn. The key finding is: the hybrid models that include Bidirectional LongTerm Short Memory (Bi-LSTM) and Convolutional Neural Network (CNN) outperform others conventional machine learning as well as deep learning models across all the datasets considered in this study, making our hypothesis valid. Research limitations Using the data from different sources and customizing the models according to each dataset, slightly decreases the usability of the technique. But, overall this methodology provides effective measures to identify the presence of sarcasm with a minimum average accuracy of 80% or above for one dataset and better than the current baseline results for the other datasets. Practical implications The results provide solid insights for the system developers to integrate this model into real-time analysis of any review or comment posted in the public domain. This study has various other practical implications for businesses that depend on user ratings and public opinions. This study also provides a launching platform for various researchers to work on the problem of sarcasm identification in textual data. Originality/value This is a first of its kind study, to provide us the difference between conventional and the hybrid methods of prediction of sarcasm in textual data. The study also provides possible indicators that hybrid models are better when applied to textual data for analysis of sarcasm.

A multi-level approach to highly efficient recognition of Chinese spam short messages

The problem of spam short message (SMS) recognition involves many aspects of natural language processing. A good solution to solving the problem can not only improve the quality of people experiencing the mobile life, but also has a positive role on promoting the analysis of short text occurring in current mobile applications, such as Webchat and microblog. As spam SMSes have characteristics of sparsity, transformation and real-timedness, we propose three methods at different levels, i.e., recognition based on symbolic features, recognition based on text similarity, and recognition based on pattern matching. By combining these methods, we obtain a multi-level approach to spam SMS recognition. In order to enrich the pattern base to reduce manual labor and time, we propose a quasi-pattern learning method, which utilizes quasi-pattern matching results in the pattern matching process. The method can learn many interesting and new patterns from the SMS corpus. Finally, a comprehensive analysis indicates that our spam SMS recognition approach achieves a precision rate as high as 95.18%, and a recall rate of 95.51%.

Introduction to this special issue on Cognitive Aspects of Natural Language Processing

Introduction to this special issue on Cognitive Aspects of Natural Language Processing

Context constraint disambiguation of word semantics by field association schemes

Word sense disambiguation is important in various aspects of natural language processing, including Internet search engines, machine translation, text mining, etc. However, the traditional methods using case frames are not effective for solving context ambiguities that requires information beyond sentences. This paper presents a new scheme for solving context ambiguities using a field association scheme. Generally, the scope of case frames is restricted to one sentence; however, the scope of the field association scheme can be applied to a set of sentences. In this paper, a formal disambiguation algorithm is proposed to control the scope for a set of variable number of sentences with ambiguities as well as solve ambiguities by calculating the weight of fields. In the experiments, 52 English and 20 Chinese words are disambiguated by using 104,532 Chinese and 38,372 English field association terms. The accuracy of the proposed field association scheme for context ambiguities is 65% higher than the case frame method. The proposed scheme shows better results than other three known methods, namely UNED-LS-U, IIT-2, and Relative-based in corpus SENSEVAL-2.